1. Field of the Invention
One or more embodiments of the invention are related to the field of image analysis and image enhancement, and computer graphics processing of two-dimensional (2D) images into three-dimensional (3D) stereoscopic images. More particularly, but not by way of limitation, one or more embodiments of the invention enable a system and method for real-time depth modification of stereo images of a virtual reality environment, for example without alteration of masks used in the initial depth augmentation process. Specifically, depth may be quickly adjusted locally and for example without the time consuming iterative workflow typical in depth conversion processes that generally requires different workgroups to adjust regions or depth in masks and send changes back for review.
2. Description of the Related Art
3D viewing is based on stereographic vision, with different viewpoints from one or more images provided to the left and right eyes to provide the illusion of depth. Many techniques are known in the art to provide 3D viewing. For example, specialized glasses may be utilized for viewing 3D images, such as glasses with color filters, polarized lenses, or anamorphic lenses. Some 3D viewing methods use separate screens for left eye and right eye images, or project images directly onto the left eye and right eye.
Virtual reality environments typically are computer-generated environments that simulate user presence in either real world or computer-generated worlds. The systems utilized to display the virtual reality environment typically include a stereoscopic display for 3D viewing and generally instrument a viewer with one or more sensors, in order to detect and respond to the position, orientation, and movements of the viewer. Based on these values, the virtual reality environment generates images to provide an immersive experience. The immersive experience may also include other outputs such as sound or vibration. Images may be projected onto screens, or provided using specialized glasses worn by the user.
The vast majority of images and films historically have been captured in 2D. These images or movies are not readily viewed in 3D without some type of conversion of the 2D images for stereoscopic display. Thus 2D images are not generally utilized to provide realistic 3D stereoscopic virtual reality environments. Although it is possible to capture 3D images from stereoscopic cameras, these cameras, especially for capturing 3D movies, are generally expensive and/or cumbersome 3D cameras. Specifically, there are many limitations with current 3D camera systems including prices and precision of alignment and minimum distance to a subject to be filmed for example.
The primary challenge with creating a 3D virtual reality environment is the complexity of generating the necessary stereo images for all possible positions and orientations of the viewer. These stereo images must be generated dynamically in approximately real-time as the viewer moves through the virtual reality environment. This requirement distinguishes 3D virtual reality from the process of generating 3D movies from 2D images as the location of the viewer is essentially fixed at the location of the camera.
Approaches in the existing art for 3D virtual reality rely on a detailed three-dimensional model of the virtual environment. Using the 3D model, left and right eye images can be generated by projecting the scene onto separate viewing planes for each eye. Computer-generated environments that are originally modeled in 3D can therefore be viewed in 3D virtual reality. However, creating these models can be extremely time-consuming. The complexity of creating a full 3D model is particularly high when it is desired to create a photo-realistic 3D model of an actual scene. This modeling effort requires that all shapes be defined and positioning in 3D in great detail, and that all colors and textures of the objects be set to match their counterparts in the real scene. Existing techniques for creating 3D virtual environments are therefore complex and time-consuming. They require extensive efforts from graphic artists and 3D modelers to generate the necessary realistic 3D models. Hence there is a need for a method for creating 3D virtual reality from 2D images.
The process of creating a 3D virtual reality is typically iterative. 2D images and possibly computer-generated elements are modeled in 3D space using depth assignments. Stereoscopic 3D images are then rendered from the 3D models and reviewed for accuracy and artistic effects. Typically, these reviews identify adjustments that are needed to the 3D models. These adjustments are applied, and the revised model is re-rendered for a subsequent review. This process may repeat several times. It is time consuming because the modifications to the 3D models and the rendering of images from these 3D models are labor intensive and computationally intensive and may be impossible in real-time. There are no known methods that provide rapid depth update and review cycles for virtual reality models. There are no known methods that allow editors to modify depth and immediately observe the effect of these depth changes on the stereoscopic images. Hence there is a need for a method for real-time depth modification of stereo images for a virtual reality environment.